(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
3 January 2003 (03.01.2003) 



PCT 



i mil iniioi n nun inn nn i u m inn mil dhiiiiii urn if ii iiiiib iiiiiiij an 

(10) International Publication Number 

WO 03/001340 A2 



(51) International Patent Classification 7 : G06F 

(21) International Application Number: PCT/US02/201 19 

(22) International Filing Date: 24 June 2002 (24.06.2002) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 

60/300,588 



22 June 200 1 (22.06.2001 ) US 



(71) Applicant (for all designated States except US): MOTION 
SENSE CORPORATION [US/US]; 39159 Paseo Padre, 
Suite 104, Fremont, CA 94538 (US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): MOSTTOV, Kirill 



[US/US]; 2029 channing Way, Apt. #1B, Berkeley, CA 
94704 (US). VERMES, John [US/US]; 39159 Paseo 
Padre, Suite 104, Fremont, CA 94538 (US). 

(74) Agent: COREN, David, J.; Fish & Richardson P.C., 500 
Arguello Street, Suite 500, Redwood City, CA 94063 (US). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, FI, GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, OM, PH, PL, PT, RO, RU, SD, SB, SG, 
SI, SK, SL, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, 
VN, YU, ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 

/ Continued on next page J 



(54) Title: GESTURE RECOGNITION SYSTEM AND METHOD 




(57) Abstract: A gesture-recognition system 
that has a plurality of inertia! sensors to generate 
inertia! data, a gesture discriminator, and two 
or more gesture interpreters. The gesture 
discriminator receives the inertial data and 
determines whether the inertial data represents 
a gesture discriminator determines that the 
inertial data represents a gesture from the 
class of gestures, whereas the second gesture 
interpreter receives the inertial data if the gesture 
discriminator does not determine that the inertial 
data represents a gesture from the class of 
gestures. 



Ptansr Motion 



(Handwriting 
Recognizer) 



TUt Motion 
Interpreter 





Token Generator 









_03001340A2_I_> 



nwsr AVAILABLE COPY 



wo 03/001340 A2 I Dill I0IIU1 II IIIIU IIIII IID III 111 Hlil Hill lilll IUII lllfl III! IIIIID llii IIII nu 



European patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, 
GB, GR, IE, IT, LU, MC, NL, PT, SE, TR), OAPI patent 
(BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, ML, MR, 
NE, SN, TD, TG). 



For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations'* appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



Published: 

— without international search report and to be republished 
upon receipt of that report 



WO 03/001340 PCT/US02/20119 



5 GESTURE RECOGNITION SYSTEM AND METHOD 

BACKGROUND 

The present invention concerns gesture recognition technology, particularly for 
mobile microprocessor-based devices with motion-sensitive input and control 
10 mechanisms. 

As mobile devices shrink, the surface area on the devices available for user input 
and control mechanisms also shrinks. Consequently, there is a growing need for control 
systems that are accurate and reliable, but do not require significant surface area on the 
device. One area of development is motion-sensing technology, such as gesture 

15 recognition technology. 

In general, two techniques have been developed for motion-sensitive input or 
control devices: "external referenced" and "internal referenced". In an "internal 
referenced" device, the sensors that permit motion-awareness are fastened, integrated, or 
otherwise mechanically coupled to the device so that both the sensors and the device 

20 become a single inertial platform. As such, these devices sense their motion over time 
with respect to a reference frame that coincides with the reference frame of the device at 
some arbitrary start time. In contrast, in an "external referenced" device, at least part of 
the sensing apparatus is fixed outside of the device's inertial frame. As such, these 
devices typically measure their position or velocity relative to a reference frame outside 

25 the device, e.g., a fixed external point. 

Internal referencing is useful for mobile devices such as cellular telephones, 
personal digital assistants (PDAs), handheld games, and cameras, since a basic 
requirement of these applications is location independence. In contrast, external 
referencing is acceptable for input devices that are connected to a desktop device, such as 

30 desktop game controllers. 

Tilt sensors and inertial sensors (e.g., accelerometers and gyroscopes) are useful 
for mobile devices using an "internal referenced" control system. Such sensors have been 
decreasing in size and cost due to the increasingly availability of micro-electromechanical 
systems (MEMS). Together with software and additional hardware, such sensors permit 

35 the user to employ gestures to control mobile electronic devices. A gesture in this case is 
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a pre-determined motion of the entire device that activates a function of the application or 
provides it with control information. 

It is well known in the art that reliable recognition of gestures is difficult. In 
addition, gesture recognition becomes increasingly difficult as the number of permissible 
5 gestures increases. Furthermore, an additional, qualitative difficulty arises when there are 
multiple classes or degrees of freedom of permissible gestures, for example planar vs. 3- 
D gestures. It should be understood that additional degrees of gesture freedom 
dramatically increase the difficulty of developing a robust recognition engine. 

A straightforward response to this problem is to limit the universe of permissible 
10 gestures, in number and especially in the number of gesture classes. For example, the 
permissible motions could be limited to one dimension of tilt, or to motions associated 
with a specific activity, e.g., musical conducting. 

More recently, attempts have been made to recognize motions from more than one 
class. For example, U.S. Patent No. 6,151,208 discloses a device that recognizes two 
1 5 classes of gestures: positional rotations of the hand as distinct from gestural rotations of 
the hand. Since a gestural rotation can be confused with multiple positional rotations, the 
device is implemented with "speculative execution of commands." With this solution, 
when the beginning of a rotation is detected, the device's state is saved. Commands 
called for by positional rotations are executed. If it is subsequently discovered that the 
20 rotation was actually a gesture, the device's state is rolled back and the function called for 
by the gesture is executed instead. 

The speculative execution solution described has the weakness that redundancy of 
processing is involved. In addition, the requirement for being able to roll back state is a 
constraint on the type of functions that can be initiated. Still further, speculative 
25 execution becomes increasingly less suitable as the number of motion classes increases. 

There is a clear requirement in the art for a method that improves the recognition 
of inertial gestures when such gestures can be from different classes. 
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SUMMARY 

In a first aspect, the invention is directed to a gesture-recognition system that has a 
plurality of inertial sensors to generate inertial data, a gesture discriminator, a first gesture 
interpreter and a second gesture interpreter. The gesture discriminator receives the 
5 inertial data and determines whether the inertial data represents a gesture from a first class 
of gestures. The first gesture interpreter receives the inertial data if the gesture 
discriminator determines that the inertial data represents a gesture from the first class of 
gestures, whereas the second gesture interpreter receives the inertial data if the gesture 
discriminator does not determine that the inertial data represents a gesture from the first 
1 0 class of gestures. The first gesture interpreter is configured to identify the inertial data as 
a particular gesture from the first class of gestures, and the second gesture interpreter is 
configured to identify the inertial data as a particular gesture from a second class of 
gestures. 

Implementations of the invention may include one or more of the following 

15 features. The first class of gestures may be linear motions, such as linear reciprocal 
motions, or planar motions. If the first class of gestures are planar motions, the first 
gesture interpreter may comprise a handwriting recognition system. The second class of 
gestures may be a tilt motion. The inertial sensors may comprise at least one gyroscope, 
accelerometer, or magneto-resistive sensor. 

20 111 another aspect, the invention is directed to a gesture-recognition system that has 

a plurality of inertial sensors to generate inertial data, a first gesture discriminator, a first 
gesture interpreter, a second gesture discriminator and a second gesture interpreter. The 
first gesture discriminator receives the inertial data and determines whether the inertial 
data represents a gesture from a first class of gestures. The first gesture interpreter 

25 receives the inertial data if the first gesture discriminator determines that the inertial data 
represents a gesture from the first class of gestures, and is configured to identify the 
inertial data as a particular gesture from the first class of gestures. The second gesture 
discriminator receives the inertial data if the gesture discriminator does not determine that 
the inertial data represents a gesture from the first class of gestures and determines 

30 whether the inertial data represents a gesture from a second class of gestures. The second 
gesture interpreter receives the inertial data if the second gesture discriminator determines 
that the inertial data represents a gesture from the second class of gestures and is 
configured to identify the inertial data as a particular gesture from the second class of 
gestures. 

3 
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Implementations of the invention may include one or more of the following 
features. A third gesture discriminator may receive the inertial data if the gesture 
discriminator does not determine that the inertial data represents a gesture from the 
second class of gestures and may determine whether the inertial data represents a gesture 
5 from a third class of gestures. A third gesture interpreter may receive the inertial data if 
the third gesture discriminator determines that the inertial data represents a gesture from 
the third class of gestures and may be configured to identify the inertial data as a 
particular gesture from the third class of gestures. A fourth gesture interpreter may 
receive the inertial data if the third gesture discriminator does not determine that the 

10 inertial data represents a gesture from the third class of gestures and may be configured to 
identify the inertial data as a particular gesture from the fourth class of gestures. 

In another aspect, the invention is directed to a gesture-recognition system with at 
least one discriminator to use inertial data to select one of a plurality of classes of gestures 
and a plurality of gesture interpreters. Each gesture interpreter is configured to identify a 

15 particular gesture from one of the plurality of classes of gestures. 

In another aspect, the invention is directed to a gesture-controlled electronic 
device that has a plurality of inertial sensors to generate inertial data, a gesture- 
recognition system, and an application that receives a token from the gesture-recognition 
system. The gesture recognition system includes a gesture discriminator that receives the 

20 inertial data and determines whether the inertial data represents a gesture from a first class 
of gestures, a first gesture interpreter that receives the inertial data if the gesture 
discriminator determines that the inertial data represents a gesture from the first class of 
gestures, and a second gesture interpreter that receives the inertial data if the gesture 
discriminator does not determine that the inertial data represents a gesture from the first 

25 class of gestures. The first gesture interpreter is configured to identify the inertial data as 
a particular gesture from the first class of gestures, whereas the second gesture interpreter 
is configured to identify the inertial data as a particular gesture from a second class of 
gestures. 

In another aspect, the invention is directed to a method of controlling a hand-held 
30 electronic device that runs an application. In the method, inertial data is generated with a 
plurality of inertial sensors embedded in the electronic device, a particular gesture is 
identified from the inertial data and a corresponding token is generated, and the token is 
sent to the application. The identifying step includes selecting one class of gestures from 
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a plurality of classes of gestures based on the inertial data, and identifying a particular 
gesture from the selected class of gestures based on the inertial data. 

In another aspect, the invention is directed to a method of recognizing a gesture by 
an electronic device. In the method, inertial data is generated with a plurality of inertial 
5 sensors embedded in the electronic device, one class of gestures is selected from a 

plurality of classes of gestures based on the inertial data, and a particular gesture from the 
selected class of gestures is identified based on the inertial data. 

In another aspect, the invention is directed to a method of recognizing a gesture by 
an electronic device. The method includes generating inertial data with a plurality of 
1 0 inertial sensors embedded in the electronic device, and determining whether the inertial 
data was generated by a gesture from a first class of gestures. If so, the inertial data is 
matched to a particular gesture from the first class of gestures, and if not, the inertial data 
is matched to a particular gesture from a second class of gestures. 

The present invention facilitates the recognition of complex inertial gestures in the 
1 5 form of intentional movements of a mobile microprocessor-controlled device. 

BRIEF DESCRIPTION OF DRAWINGS 
FIG. 1 illustrates a mobile microprocessor-based electronic device. 
FIG. 2 is a schematic block diagram illustrating information flow through a 
20 mobile electronic device. 

FIG. 3 is a block diagram illustrating the components of a parser according to the 
present invention. 

FIG. 4 is a flow-chart representing the method performed by the parser of FIG. 3. 
FIG. 5 is a schematic diagram of a gesture recognition system for a mobile 
25 electronic device. 

FIG. 6 is a block diagram illustrating the components of a parser according to an 
alternative implementation of the present invention. 



DETAILED DESCRIPTION 

In prior art, sensor data is presented directly to a recognition system block for 
processing. In contrast, in the present invention, a classification step takes place first, 
dividing the recognition problem into two tasks: 

(1) Assign gesture data according to broad criteria to one of a plurality of 
predetermined classes 

5 
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(2) Interpret said gesture using the recognizer specifically designed for the 
assigned gesture class 

In the gesture classification step, the incoming stream of sensor data is examined 
for gross criteria, for example planarity or linearity, rather than for gesture identity. It 
5 should be noted that such criteria are not limited to these examples, but could refer to any 
criteria that differentiate gesture classes rather than gestures themselves from one another. 
The task of recognition is thus simplified, since recognition is carried out on pre- 

classified gestures by specialized recognizers specific to a given gesture's class. Also, 

> 

advantageously, improvement of such specialized gesture interpreters is easier to carry 
1 0 out than improvement of the general recognizers present in prior art. 

A microprocessor-based mobile electronic device 10 in which the invention can 
be implemented is illustrated in FIG. 1 . The present invention provides a method and a 
system for recognizing gestures performed by the user of the electronic device. The 
device can be a personal digital assistant (PDA) as shown, or it can be some other hand- 
1 5 held device, such as a cellular phone, one-way or two-way pager, hand-held game or 
camera. Of course, if the device is a cellular phone, it can send and receive radio- 
frequency telephone signals, and if the device is a PDA, it can be connected to a docking 
station for synchronization with a desktop computer. However, beyond these types of 
situations, the device is generally mobile and, for ordinary operation, need not be 
20 physically connected to or communicate with a stationary computer system or sensor 
system. 

The electronic device 10 includes a gesture recognition system that permits the 
user to gesture with the device to input commands or data. A gesture in this case is a pre- 
determined motion of the entire device (as opposed to manipulation of a control piece, 

25 such as a button). The gesture can activate a function of an application or provides it with 
control information. Such gestures are intended to provide the electronic device with user 
interface input similar in purpose to input from conventional user interface devices, such 
as buttons, keys, styli/digitizers, and the like. 

The information flow in the gesture-controlled electronic device 10 is illustrated 

30 in FIG. 2. The electronic device 10 includes one or more inertial sensors 12 that generate 
analog or digital signals 14 representing the motion or position of the electronic device 
10. The inertial sensors can be fastened, integrated, or otherwise mechanically coupled to 
the device so that both the sensors and the device become a single inertial platform. As 
such, these devices sense their motion with respect to the reference frame of the device 
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itself. In contrast to a position sensor that measures a position with respect to a fixed 
reference sensor system, the inertial sensors 12 detect the motion of the device (e.g., 
velocity, acceleration, angular velocity, or angular acceleration). In addition, the inertial 
sensors can be entirely internal to the electronic device 10, so that the inertial data can be 
5 generated without requiring data from an external sensor or the exchange of data with an 
external processor. 

The sensors can be micromachined electromechanical systems (MEMS), such as 
accelerometers, gyroscopes, or magneto-resistive sensors. Assuming that accelerometers 
are used, the sensors can measure the acceleration of the device with respect to a 

1 0 reference frame that coincides with the reference frame of the device at some arbitrary 
start time. Similarly, assuming that gyroscopes are used, the sensors can measure the 
angular acceleration of the device with respect to a reference frame that coincides with 
the reference frame of the device at some arbitrary start time. On the other hand, 
assuming that magneto-resistive sensors are used, the sensors can measure the angular 

1 5 position of the device with respect to the local magnetic field. 

In one implementation, the electronic device includes three sensors (e.g., 
accellerometers) to measure motion along three perpendicular linear axes (the X, Y and Z 
axes), and three sensors (e.g., gyroscopes or magneto-resistive sensors) that generate data 
from which motion of the device about three perpendicular spin axes may be deduced. 

20 Typically, the three linear axes are parallel to the three rotational axes. In addition, as 
shown by FIG. 1, the sensors can be oriented so that the measurement axes generally 
align with the gross shape of the electronic device. For example, a first linear axis and a 
first rotational axes may be aligned with the primary longitudinal axes of the electronic 
device, and a second linear axis and a second rotational axes may be aligned with the 

25 secondary longitudinal axes of the electronic device. 

Returning to FIG. 2, assuming that the inertial sensors 12 generate analog signals, 
these analog signals are fed to a transducer 16, such as an analog to digital converter 
(ADC), to generate digital signals 18. 

The digital signals are then fed to gesture recognition system 15 that includes a 

30 signal conditioning mechanism 20 and a parser 24. The signal conditioning mechanism 
performs data "cleaning" to reduce noise in the signal, generating a cleaner digital signal 
22. 

The digital signal 22 is fed to the parser 24, discussed in further detail below, that 
extracts semantic information from the inertial data. The parser 24 can identify gestures 

7 
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in a variety of classes from the inertial data. The gestures be either simple, e.g., circular 
motion or shaking, or complex, e.g., tracing of letters or numbers. Once the parser 24 
identifies the gesture, it generates a token 26 that is directed to an application 28 running 
on the electronic device 10. The tokens generated by parser 24 represent specific 
5 gestures, but the command or data represented by the tokens are left undefined. Instead, 
the tokens are interpreted by the application 28 (although the tokens might have default 
interpretations given by an operating system or by the parser, e.g., an "x-gesture" is . 
interpreted as the keystroke Y). This permits different applications to assign different 
actions or meanings to the tokens. For example, one application can assign a token for a 

10 shaking motion to a command to close the application, whereas another application can 
assign a token for a shaking motion to a data input, such as a letter or number. In an 
event-driven environment, the token 26 can be considered an event, much like a mouse- 
click or a keystroke, with the response controlled by the methods of the objects in the 
application. Optionally, if the gestures belong to a subset of the classes, or the class of 

1 5 gestures cannot be classified by the parser, then the sensor data may be fed directly to one 
of the applications for use by the application. 

The treatment of tokens by the electronic device can be contrasted with electronic . 
devices in which positional or inertial data is fed directly into an application. For 
example, in conventional game control devices, the application is programmed to map the 

20 incoming data to a scalar control parameter, e.g., a position on a display, or to perform a 
preset action in response to a preset sequence of inertial data. In contrast, the tokens 
generated by parser 24 represent specific gestures, but the command or data represented 
by the tokens are left undefined. 

The basic functional blocks of one implementation of the parser 24 are illustrated 

25 in FIG. 3. The parser includes a discriminator 30 that decides which class of gesture is 
represented by the inertial data, and two interpreters 32, 34 that match the inertial data to 
a particular gesture in the class that was selected by the discriminator 30. Possible classes 
of gestures include tilting, linear motions, reciprocal motions and planar motions. 

The equivalent method performed by the parser is illustrated in FIG. 4. First, the 

30 parser determines what class of gesture is represented by the inertial data. If the parser 
determines that the inertial data represents a gesture from a first class, the parser matches 
the inertial data to a particular gesture from the first class. Similarly, if the parser 
determines that the inertial data represents a gesture from a second class, the parser 
matches the inertial data to a particular gesture from the second class. 

8 
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As noted in the background, one problem in prior art inertial input devices is that, 
given multiple degrees of freedom and the possible complexity of gestures, matching the 
inertial data to a particular gesture is difficult and unreliable. However, by initially 
dividing the inertial data into different classes with the discriminator 30, each recognizer 
32 can be optimized for the particular class of gestures. This improves the reliability and 
speed of the gesture-recognition process. Consequently, the discriminator and recognizer 
can make a final determination that the inertial data represents a particular gesture from a 
particular class. Thus, the device need not save a series of states, or use a complex roll- 
back system to undo a state change resulting from an incorrect classification of the 
gesture. 

Referring to FIG. 5, in one implementation, the mobile device includes three 
accelerometers 40 and three magneto-resistive (MR) sensors 42. The accelerometers are 
positioned such that their sense axes are mutually perpendicular. Likewise, the MR 
sensors are positioned with mutually perpendicular sensitive axes. Analog signals from 
each sensor 40 and 42 is directed to an associated ADC 44. Digital signals from the ADC 
44 are directed to a preprocessor 46 for preliminary data processing of the signal, such as 
calibration, filtering and scaling. 

The processed data from the preprocessor 46 is held by a sensor data memory 48. 
For example, the data memory can be a FIFO buffer, and the data from the preprocessor 
46 can be sampled at regular intervals and placed into the FIFO buffer. Other portions of 
the gesture recognition system can request data and be presented with sextuples of digital 
data, with each of the six parameters corresponding to an output of one of the six sensors. 

In one embodiment, the parser is configured to identify gestures from two classes 
of permissible gestures: 

(a) translation motion embedded in a plane without angular motion, and 

(b) angular motion without translation, for example a tilting motion. 

To perform the gesture recognition, data from the memory 48 is directed to a 
linear or planar motion discriminator 50. If the discriminator 50 detects linear or planar 
motion, the motion data is transferred to a planar gesture recognizer 52, such as a 
conventional handwriting recognition system. On the other hand, if the discriminator 50 
does not detect linear or planar motion, the motion data is transferred to a tilt gesture 
recognizer that determines the direction and degree of tilt of the device. 

In a second implementation, the parser is configured to identify gestures from two 
classes of permissible gestures: 
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(a) reciprocal motion in three dimension, and 

(b) angular motion without translation, for example a tilting motion. 
In the second implementation, if the discriminator 50 identifies a linear or 

reciprocal motion, the inertial data can be fed to the first interpreter 52. The first 
5 interpreter 52 identifies the particular reciprocal motion, e.g., a motion along one of the 
six linear semi-axes (+X, -X, +Y, -Y, +Z, -Z). Since the interpreter 52 is dedicated to a 
limited type of motion, it can be designed to identify these reciprocal motions with 
greater accuracy. 

On the other hand, if the discriminator 50 does not identify a reciprocal motion, 

10 the inertial data can be fed to the second interpreter 54. The second interpreters 54 

identifies a direction and degree of "tilt", e.g., the amount by which the electronic device 
is tilted away from the gravitational vector along one of the six semi-axes. 

Different classes of gestures can be used for different types of functions by the 
applications. For example, reciprocal motions can be interpreted as commands, e.g., to 

1 5 open or close an application, whereas tilt motions can be interpreted as cursor control 
data, e.g., selection of an item from a list or positioning of a cursor on a screen. 

The discriminator 50 can identify linear motion, particularly "reciprocal motions", 
i.e., short motions of the electronic device along an axis and then back to its starting 
point. The identification of the gesture is passed from the interpreter 52 or 54 to a token 

20 ID generator 56 which generates a token identifying the type of gesture. The token from 
the generator 56 is then passed to a processor which interprets the token as a control 
command or a character (digits, letters, special characters). 

Referring to FIG. 6, in another implementation, a parser 24' is organized with a 
hierarchy of discriminators 60, 62, 64. The first discriminator 60 determines whether the 

25 gesture belongs to a first class of motions. If the first discriminator recognizes the 
gesture, the inertial data is fed to a first interpreter 70 that is configured to identify 
particular gestures from the first class. If the motion does not belong to the first class, 
then the inertial data is fed to the second discriminator 62. The second discriminator 62 
determines whether the gesture belongs to a second class of motions. If the second 

30 discriminator recognizes the gesture, the inertial data is fed to a second interpreter 72 that 
is configured to identify particular gestures from the second class. If the motion does not 
belong to the second class, then the inertial data is fed to the third discriminator 64. The 
third discriminator 64 determines whether the gesture belongs to a third class of motions. 
If the third discriminator recognizes the gesture, the inertial data is fed to a third 

10 
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interpreter 74 that is configured to identify particular gestures from the third class. If the 
motion does not belong to the third class, then the inertial data is fed to a generic fourth 
interpreter 76 that is configured to identify gestures that do not belong to the other three 
classes. 

5 For example, the first class of gestures can be reciprocal linear motions, the 

second class of gestures can be tilting motions, the third class of gestures can be planar 
motions, and the fourth class of gestures can be arbitrary three-dimensional motions. Of 
course, the ordering of these classes of gestures can be changed. 

Of course, the parser 24 can include just two discriminators, or four or more 
10 discriminators. However, in general, each discriminator is configured to recognize a 
specific class of gestures, and each discriminator is associated with an interpreter that 
identifies specific gestures in the class. With this functional organization, each 
discriminator can be narrowly tailored to accurately identify the specific class of gestures. 
Similarly, each interpreter can be narrowly tailored to accurately identify gestures in a 
15 specific class. Because the discriminators and interpreters are narrowly tailored, they can 
be more accurate than general-purpose template matching algorithms. 

The discriminators and interpreters can be implemented as hardware, software or 
firmware, or a combination of hardware, software or firmware. 
What is claimed is: 
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1 . A gesture-recognition system, comprising: 

a plurality of inertial sensors to generate inertial data; 

a gesture discriminator that receives the inertial data and determines whether the 
inertial data represents a gesture from a first class of gestures; 
5 a first gesture interpreter that receives the inertial data if the gesture discriminator 

determines that the inertial data represents a gesture from the first class of gestures, the 
first gesture interpreter configured to identify the inertial data as a particular gesture from 
the first class of gestures; and 

a second gesture interpreter that receives the inertial data if the gesture 
1 0 discriminator does not determine that the inertial data represents a gesture from the first 
class of gestures, the second gesture interpreter configured to identify the inertial data as a 
particular gesture from a second class of gestures. 

2. The system of claim 1, wherein the first class of gestures comprise planar 
motions. 

3. The system of claim 2, wherein the first gesture interpreter comprises a 
handwriting recognition system. 

4. The system of claim 1 , wherein the first class of gestures comprise linear 
reciprocal motions. 

5. The system of claim 1, wherein the second class of gestures comprises a tilt 
motion. 

6. The system of claim 1, wherein the inertial sensors comprise at least one 
gyroscope, accelerometer, or magneto-resistive sensor. 



15 



20 



25 



7. A gesture-recognition system, comprising: 
30 a plurality of inertial sensors to generate inertial data; 

a first gesture discriminator that receives the inertial data and determines whether 
the inertial data represents a gesture from a first class of gestures; 

a first gesture interpreter that receives the inertial data if the first gesture 
discriminator determines that the inertial data represents a gesture from the first class of 

12 
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gestures, the first gesture interpreter configured to identify the inertial data as a particular 
gesture from the first class of gestures; 

a second gesture discriminator that receives the inertial data if the gesture 
discriminator does not determine that the inertial data represents a gesture from the first 
5 class of gestures and determines whether the inertial data represents a gesture from a 
second class of gestures; and 

a second gesture interpreter that receives the inertial data if the second gesture 
discriminator determines that the inertial data represents a gesture from the second class 
of gestures, the second gesture interpreter configured to identify the inertial data as a 
1 0 particular gesture from the second class of gestures. 

8. The system of claim 7, further comprising: 

a third gesture discriminator that receives the inertial data if the gesture 
discriminator does not determine that the inertial data represents a gesture from the 
15 second class of gestures and determines whether the inertial data represents a gesture 
from a third class of gestures. 

9. The system of claim 8, further comprising: 

a third gesture interpreter that receives the inertial data if the third gesture 
20 discriminator determines that the inertial data represents a gesture from the third class of 
gestures, the third gesture interpreter configured to identify the inertial data as a particular 
gesture from the third class of gestures. 

10. The system of claim 9, further comprising: 

25 a fourth gesture interpreter that receives the inertial data if the third gesture 

discriminator does not determine that the inertial data represents a gesture from the third 
class of gestures, the fourth gesture interpreter configured to identify the inertial data as a 
particular gesture from the fourth class of gestures. 



30 11. A gesture-recognition system, comprising: 

at least one discriminator to use inertial data to select one of a plurality of classes 
of gestures; and 

a plurality of gesture interpreters, each gesture interpreter configured to identify a 
particular gesture from one of the plurality of classes of gestures. 
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12. A gesture-controlled electronic device, comprising: 

a plurality of inertial sensors to generate inertial data; 

a gesture-recognition system that includes a gesture discriminator that receives the 
5 inertial data and determines whether the inertial data represents a gesture from a first class 
of gestures, a first gesture interpreter that receives the inertial data if the gesture 
discriminator determines that the inertial data represents a gesture from the first class of 
gestures, the first gesture interpreter configured to identify the inertial data as a particular 
gesture from the first class of gestures, and a second gesture interpreter that receives the 
1 0 inertial data if the gesture discriminator does not determine that the inertial data 
represents a gesture from the first class of gestures, the second gesture interpreter 
configured to identify the inertial data as a particular gesture from a second class of 
gestures, and generates a token representing the particular gesture; and 

an application that receives the token from the gesture-recognition system. 

15 

13. A method of controlling a hand-held electronic device that runs an application, 
comprising: 

generating inertial data with a plurality of inertial sensors embedded in the 
electronic device; 

20 identifying a particular gesture from the inertial data and generating a 

corresponding token, the identifying step including 

selecting one class of gestures from a plurality of classes of gestures based 
on the inertial data, and 

identifying a particular gesture from the selected class of gestures based on 
25 the inertial data; and 

passing the token to the application. 

14. A method of recognizing a gesture by an electronic device, comprising: 
30 generating inertial data with a plurality of inertial sensors embedded in the 

electronic device; 

selecting one class of gestures from a plurality of classes of gestures based on the 
inertial data; and 
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identifying a particular gesture from the selected class of gestures based on the 
inertial data. 

15. A method of recognizing a gesture by an electronic device, comprising: 
5 generating inertial data with a plurality of inertial sensors embedded in the 

electronic device; 

determining whether the inertial data was generated by a gesture from a first class 
of gestures; 

if so, matching the inertial data to a particular gesture from the first class of 
10 gestures; and 

if not, matching the inertial data to a particular gesture from a second class of 
gestures. 
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